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ABSTRACT 

This invmtion provides a method to compress digital moving pictures or video signals 
based on an overcomplete basis transform using a modified Matching Pursuit algorithm. 
More particularly, this invention focuses on the efficient coding of the motion residual 
picture, which is generated by the process of motion estimation and comprasation. A 
residual energy segmentation algorithm (RESA) is proposed to obtain an mitial estimate 
of the shape and position of high-energy regions in the residuaL A progressive 
elimination algorithm (PEA) is proposed to reduce the number of matching evaluations in 
the matching pursuits process. RESA and PEA speed up the encoder by many times for 
finding the matched basis from the pre-specified overcomplete basis dictionary. Three 
parameters of the matched pattern form an atom, which defines index (into the 
dictionary) and position of selected basis, as well as the inner product between the chosen 
basis pattern and the residual signal. This invention proposes a novel atom position 
coding method using quad tree like techniques and a novel atom modulus quantization 
scheme. A simple and efficient adaptive mechanism is introduced for the quantization 
and position coding design to allow the invented system to operate properly in low, 
medium and high bit rate situations* These new algorithm components result in a much 
faster encoding process and better compression performance over previous matching 
pursuit based video coders. 
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BACKGROUND OF THE INVENTION 

A sequence of pictures can occupy a vast amount of storage space and require 
very high transmission bandwidth when represented in an uncompressed digital fonn 
Pomt to point digital video communication became practicable several yeais ago 
following advances in computer networks and signal compression technology. 

The standardization effort for digital video compression was initiated in 1988 
Currently, the Moving Picture Experts Group (MPEG) committee under ISO/IEC has 
completed both MPEG-1 and MPEG-2 standards: the MPEG-4 standard has also been 
complied, but new proposals are still being accepted. CCITT developed a series of 
recommendations - H.261. H.263 and H.263+ - that focus on low bit rate applications. 
All of these att^pts at standardization utilize a two-step pioceduie to compress a video 
sequence. The first step uses a motion estimation and compensation algorithm to create a 
predicted video frame for die current video frame using the previous video frame - the 
difference between the current video frame and the predicted video frame is computed 
and is called the moticMi residual picture (MRP). The second step in the standard 
procedure is to code die MRP using the Discrete Cosine Transform (DCT). Such DCT- 
based systems do not perform well in all circumstances. At the low bit rates needed for 
personal video communication. DCT-based systems cause noticeable distortion and 
visible block artifacts. For high visual quality applications, sudi as DVD. the 
compression ratio achieved is quite low. 

Motion residual pictures can be coded using other transform-based techniques For 
example, discrete wavelet transforms (DWT) and overcompleie basis transforms can also 
be used. Zakhor and Neff presented a motion residual coding system in U.S. Patent 
5699121 based on an ovcaromplete basis transform algorithm called matching pursuit* 
this was first proposed by Mallat and Zhang in IEEE Transaction in Signal Processing 
vol. 41, No. 12. Dec. 1993. Zakhor and Neff's video coder improves botii tiie visiml 
quality and the PNSR over standard DCT-based video codeis. However. Uieir system is 
very slow and tiie compression performance is not optimized due to an ad-hoc design for 
matched basis position coding and quantization of the Iransfoim coefficients. Thus it is 
valuable to unprove flje overcomplete transfcmn based video coding technique in both 
speed and efficiency. 



SUMMARY OP THE INVENTION 

TTie current invention is an improved coder for overcomplete-transform based 
residual picture coding, used for motion compensated video compression systems. This 
invention is analogous to previous matching puisuit video coders in tfiat ttiey decompose 
the residual image into a list of atoms, which represent basis functions from an 
overcomplete dictionary. However, the atom finding process is accelerated using the 
Residual Energy Segmentation Algorithm (RESA) and the Progressive Elimination 
Algorithm (PEA). The basis dictionary is often very large in order to characterize the 
features appearing frequently in motion residual images. To find an atom, prior matchmg 
pursuit mettiods have performed an exhaustive search of ttie basis dictionary; this 
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^sT?r„fSlf J ^ '^"""^ ^'i^' ^''^^'^ basis in the dictionanr 

RESA idenufies the approximate shape and position of regions with high energy TSe 
motion residual .mages such that a good match can be found by compaS wTth a 
smaller subset of bases within the dictionary. Furthermore. PEA progressTyTre^^t^ 
pattern candidates from consideration by pre^mpuU,^ the energ? of^rch wiSI - 
Si'^^S: '^T^'^u "^rr^" to find the Zi match. Wh^v^ a 

Ttom 1? ; 'n-'^'f "P*'*^ '^'""^'"e characterized by^ 

atoinjhe foregoing steps of finding atoms and updailiig residuals ai« rtspeated untU the 
desired compression bit rate or quality has been achieved. unm me 

The invention introduces a new modulus quantization scheme for matching 
pur^it with an overeomplete basis, which changes the atom finding procedure. The 
coefficients produced directly from the transform are continuous floating-point valu^ 
which require quantization for optimal digital coding under a bit budget In the matchinE 
pursuit algontiim. it is necessary to use an in-loop quantizer - where each found atom ^ 
firat quantized, and then used to update the residual image. Thus each atom affects the 
selection of subsequent atoms. If the quantizer is specified before coding begins as in 
previous matchmg pursuit methods, it is difficult to optimize the quantizidnlch^ne as 
tfie optimal quaiitizer design depends on statistics of tiie list of chosen atom moduli. The 
disclosed quantization scheme chooses tiie quantizer adaptively during the atom 
searchmg process. j b «uiii 

In addition to ttie atom modulus, the index of the chosen basis and the position of 
the atoms need to be transmitted in an overcomplete-transfbrm based coder The 
invention mcludK a method to code the atom position infoimation efficiently. W atom 
position distribution forms a 2D map. where pixel values of one and zero represent the 
presence or lack thereof of atoms in each position respectively. A quadtree like technique 
^^iriZ, °P.!? *° the position map. The modulus and basis index infbmiation 
are embedded m the position coding. The atoms for different channels of color video (Y 
f° independently. Previous systems anange all the atoms as a ID list, and 
code the atom parameters one by one. It is hard to use the context information between 
atom positions and the special properties of atoms in different color chamiels for efficient 
codmg by considering the list of all atoms as a ID signal. ««««« 

AU atom parameters are transmitted after they have been encoded into a 
compressed version of the residual pictures. The associated decoder is also presented in 
^^H^wrc^' i^"^^' reconstructs the residual picture through interpreting the 
coded bit sti:eam back into atom parameters and combining the atom information to form 
tiie reconstructed stremn of residual images that are then combined with the moti^ 
compensated unage to form the reconstructed video stream. 

In summaiy. the invention is a method for encoding motion residual pictures 
consisting of the steps of 1. forming tiie atom decomposition of the residual unage ta^ 
overcomplete basis space usmg the modified matching puisuh algorithm. 2. choosing the 
modulus quanuzer, 3. codir^ the atom position mJp. modulus as w;ir^srdel for 
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S?i^v^"r "^"^ *« tav. been «„:odecl ustog 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 illustrates the overall diagram of video compression systems that use the over- 
complete basis transform and associated coding methods of the invention 
FIG. 2 IS an example of a motion residual picture processed by the Invention. 
FIG. 3 illustrates a simple dictionary ^th 16 bases. 

FIG. 4 describes the whole atom decomposition process based on over-complde basis. 
(RESA) ^ s^mentation algorithm 

FIG. 6 illustrates the first step of RESA. 

HG. 7 illustrates the second step of RESA: the horizontal growing scheme. 
FIG. 8 dlustrates the third step of RESA: the vertical growing scheme 

^gOT'L^Pm. *^ '^'^^^ ""^'^ *^ progressive elimination 

roSdi'caSS. *° ^ '•'^""'^'^ ^ --h^ns 

FIG. 11 illustrates the fast calculation of region energy. 

FIG. 12 illustrates the parameters for one atom. 

FIG. 13 is an example of an atom position map. 

FIG. 14 is a flowchart illustrating the atom encoding process. 

FIG. 15 is a flowchart illustrating the decoding of compressed residual signal. 

draiinj'^'^ numerals refer to corresponding parts throughout the several views of the 

DETAILED DESCRIPTION OF THE INVENTION 

FIG. 1 aiusttat^the associated processing executed by a video compression apparatus 10 
that employs the invented residual picture coder 20. The video frame is initiaUy 
processed by a motion estimator 30, which compares the current fhime with one or two 
reference frames. In most cases, objects in the video change their posiUon in successive 
frames whde the background remains the same. Since thi reference frames havrbein 
transmitted to video decoder 12, some regions in the reference frame can be used to 

^ i"^^' ^ ^^^'^'^ regions within the 

reference firames that are similar to r^ons within the currem frame. The motion 
compensator 32 produces the difference between those similar regions and combines 
them as a motion residual picture. The position relationships between similar regions are 
represented as motion vectors, which are processed by motion vector encoder 34. The 
atom decomposer 40 processes the residual picture first, and then the atom encoder 42 
oTK^^ the resulted atoms. The coded motion vectora and atoms are combined into 
one bit stream by die multiplexer 22. The compressed video is transmitted or stored by 
apparatus 24. which wUl deliver the video in compressed format to the video decoder 12 
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The lower part of Fia 1 illustrates the decoder 12. In which the demuhiplexer 26 
separates comprised video signal, sending corresponding bits to thTmoZ Zc^r 
decoder 36 and the r«;,dual picture decoder 28 i^ectively. T^e motion re^SScLTS 
forms the prediction frame from the reference frame and the motion vecto^e ta^JiS 
residual decoder 28 reconstructs the residual. These two signals are add«l togerherl^ 
generate Uie final reconstructed video frame. HO. 1 shows that residual okW? al^ 
part of video compression. wunig is a core 

^1 u ^'Otion 'esidual picture for the Y color channel. TTie orieinal 

residual has both negative and positive values. For proper displayi^g of tte re£ 
signal as a 256 level gray image, the pixel values in the residual aVe sh^e^aTscafid so 
that pure gray mea,« zero. whUe black and white represent negative and po^tive vlS 
respccuve^y. Note that the residual picture contains several wS 
correspond to the motion of objects in the video. regions, which 

Most signal compression techniques transform the original data into some moie compact 
format through drfferait kinds mathematical tnnsformatkms. Some n^^ZS 

SZi r* ^"^l"^ » ^Wch forms an Swe 

^ansforaiatton matrix Recently, overcomplete basis and associated transfomution 
algontfuns have received considerable attention. The number of bases in an overS?ete 
basis dictionaor is much larger than dimensfon of the original data. The benefiToflS 
overconvlete basis is that the transformed coefficients are more effective in rewLnti^^ 
Oie toie features in the original signal. There exist many mathematicS meti^TbuiW f 
basis dictionajy for different signals. Several dictionaries for video motion rSdual 
pictures have been dijsigned and proved to cover features in residual pictSS^wS^ For 
NeTSld *'^°"«'y,''«««lon separable Gabor functions has teen desSd ^ 

WPP *" ^"^^ °» Matching Pureuits\ 

IEEE Transactions on Circuits and Systems for Video Technology. Feb. 1997 158-171 
^''^^o^.^y based on Haar functions has been described by Vleeschiuwer and 
Macq in "New dictionaries for matching pursuit video coding", ftoc of Z lISS 
International Conference on In«^ge Processing, vol. 1. 764-768. FIG. 3 is a simple 
example dichona^y containing 16 bases. Any of these dictionaries can be us^ ^1^5 
mvention. Focusing on the above-mentioned Gabor dictionary, there S 4^0 2^ 
functions explicitly mentioned. However, it actually includes m^ more basiVstmcti,^ 
imphciUy since each of those 400 2D functions can be placed at lyery possible pSS 
witiiin the .mage. Using a frame size of 176X144 pixels implies ThaVSe dlS^ 
aaually contains 400X176X144=5.7 million bases s^ctures - ..hiTmTJTS 
r' wP'f c^x^S^*^""**'^" directly using the "matching purTuU algorithm" 

uictionanes , IEEE Transaction in Signal Processing, vol. 41. No. 12. Dec. 1993 will 
Jfce an exfremely large mmiter of computations to determine tiie transfonn coefficieTte 
SI9T2 rl!? ''"T 1^'° ^^"'P'^'^''". '"vented by Zakhor and Neff i^^ pSl 
LvenS^L ''^^^^^^ " Computationally expeSsS 

The mvention focuses on the way to transfomi residual pictures based on general 

t^Z2Z^'" f'^T^ decomposer 40. and Oie way t^oSe ^e 

transfonned coefficients, which is the task of the atom encoder 42. 
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•n»e operation of the atom decomposer 40 is fully described in FIG 4 The first 
step (block 61) executed by the atom decomposer 40 is fo find the initial search region 
Ul^f f ^^^^ *® residual energy segmentation algorithm (RESA) shown iii 
FIG. 5. RESA IS based on a gen^ r^on growuig idea. It initially selects a 2x2 blodc as 
start pomt for region growing (block 70). This step requires the division of the residual 
picture mto 16x16 blocks, as shown in FIG. 6. The energy, which is the sum of the square 
of all pixel mtensities, is computed for each block, and the block with highest energy is 
found, as block 71 shown in FIG. 6. Block 71 is further divided into four 8x8 sob-blocks 
and the sub-block 72 with the highest energy is found. Within that 8x8 sub-block 72. the 
bluest energy 2x2 block 73 is found - this block will be used as the start point for reeion 
growing. * 

The next step of RESA (block 74 in FIG. 5) is to check the 2x2 block in the left side of 
the current region. FIG. 7 illustrates this step of RESA. A threshold is calculated 
dynamically as 

T= AE*max(7-AU, 5)/10 
where AU is the number of blodcs that have been added on die left side of the start block, 
and AE is the avo^ge energy per 2x2 blodc of current region. If the energy of die 
checked 2x2 block is larger dian the current direshold, the tested 2x2 blodc is grouped 
wiOi die current region together to form a new laiger current region. Oth^ise, a stop 
point has been found on this side, and we do not group the blocks together. In a similar 
symmetric feshion. check die 2x2 block on Uie right side of die current region. Continue 
growing left and right side iteratively. until stop points ate found on both sides or the 
widtii of the rectangle has reached 32 (whichever comes firat). A horizontal strip 
rectangle 75 is formed after this step, the dimension of the strip is 2*2m, l<=m<=16. 

The final step of RESA (block 76 in FIG. 5) U to grow die region vertically based on 
strip 75, as shown in FIG. 8. Assume die widfli of die strip 75 is W. Consider die 2*W 
strip rectangle above the current region, and a threshold 

Ts= AEs*max(7-AUs. 5)/10 
Where AUs is die number of 2*W rectangles diat have been added above die initial strip 
and AEs is die average energy per 2*W rectangle included in die current region. If die 
tested 2*W rectangle has an energy diat is larger dum a dueshold. merge it into current 
region. Otherwise, a stop point has been found on this side. In a similar, symmetric 
fashion, check the 2*W rectangle below die current region. Continue growing up and 
down iteratively until stop points are found on boUi sides or the height of die cun^ent 
region has reached 32 (whidiever comes first). In die end we get a rectangle 77 diat has 
dimension 2n*2m, l<=n.mosl6. 

Returning now to PIG. 4, the process for finding die closest matched basis from the giv«i 
dictionary is iUustrated (block 62). The degree of matching between a basis and die 
residual picture is represented by die absolute value (modulus) of dieir inner product, 
which is called die atom modulus. A large modulus implies a good match. The process of 
deteraiining diis modulus requires computing a number of inner products, and selecting 
die one widi die largest modulus as die cunrent atom. This process is the slowest pan in 
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the matching pursuit algorithm. In the classical matching pursuit algorithm, the inner 
product between residual and each of the millions of elements in dictionary would need 
to be computed to determine the modulus. In the prior art, the 16*16 block with the 
highest energy in the residual picture is simply selected as the initial seaidi i^ion - eadi 
basis structure is centered at each location in the chosen block, and the inner product 
between the basis structure and the corresponding residual region will be computed. For a 
dictionary with 400 basis, this process requires 256x400=102400 inner pioduct 
calculations. FIG. 9 illustrates the modified fast matching pursuit process. 

The resulting RESA rectangle 77 in FIG. 8 provides an initial estimation for the shape of 
the high-energy feature. It is used to filler out bases in the dictionary that have a shape 
that is too different from the RESA rectaiigle. A subset of matching basis candidates 
(block 80) is then formed. Assume the width and height of rectangle 77 is w and h 
respectively. A sub-dictionary is formed containing all bases with shq>e5, specified by 
width and height respectively, that satisfies: 

w-twl<=width<=w+tw2 and h-thl<=*eight<=h+th2 
where twl,tw2,thl and th2 arc some values to confine the basis size. These values may be 
changed and adjusted according to the dictionary stmcture. The laigest and smallest sizes 
of tested bases are illustrated as rectangle 90 and 91 in FIG. 10, Block B80 is a simple 
sub-dictionary example containing four bases. 

RESA also estimates the location of high-energy features in the residual picture. The 
position candidates for matching bases are selected around the center of the RESA 
rectangle 77 (block 81). FIG. 10 shows a small rectangle 92 whose center is the same as 
RESA rectangle 77. It is supposed that all pixels within rectangle 92 will work as a center 
for the tested residual region. Rectangle 94 in FIG. 10 is an example whose center is 
point 93, or left-top comer of rectangle 92, The width (ws) and height (hs) of lectangle 92 
is supposed to be variable with RESA rectangle 77. The relationship is 

ws=: 2*min(w/2 + 1.6) and hs=2*min(h/2+1.6) 
Hie size of rectangle 92 can be decided by other rules or simply be fixed in an 
implementation. The basic idea is that a good match is located around the center of the 
RESA rectangle 77. Furthermore, any positions within rectangle 92 that already contain 
the center of an atom will not be considered for any new atoms. Point 95 in FIG. 10 is an 
example. The prior art does not place such a restriction. The idea is that if one atom 
provides a good fit. it should remove the energy around its center without introducing too 
much extra energy at its boundary. It makes no sense for the matching pursuit algorithm 
to return to the same position to produce a second atom. However, it does occur in 
practice because of bad pattern fitting and quantization. This restriction of forcing no 
position repetition has almost no effect on coding performance and makes the coding of 
the atom position information simpler. 

The next processing step (block 89 in FIG. 9) is called the progressive elimination 
algorithm (PEA) for the residual matching pursuit. It is independent of the method used 
to form the testing basis sub-dictionary and set of testing positions. For example, PEA 
will still operate if the sub-dictionary is the entire dictionary, and set of position 
candidates is the set of coordinates comprising the whole residual picture. PEA is a 
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method of Hnding the closest basis match more efficiently by progressively ronoving 
comparison candidates from consideration. This contrasts with classical matching puisui* 
which compares all basis candidates at all possible positions. InitiaUy the maximum 
modulus Mm is set to be zero (block 82). Next a basis b(k.l) is considered (block 83) 
wh»e k and I represents the width and height of the 2D basis function. A same sized 
region centered at one position candidate r(k,I.p) in residual picture is formed (block 84) 
Block 85 compares |lr(k.I,p)||. the energy of r(k,l.p). with cunent maximum modulus 
(Mm) to decide if there is a need to calculate the Inner product between r(k,l.p) and b(k,0. 
In orda- to explain this operation, recall the mathematical triangle inequality: 

|<r(k,l.p),b(k,l)>| <= ||r(k.l.p)I| ||b(k,l)|| 
Hie objective of matdiing pursuU is to find the maximum |<r(k.l.p).b(k.l)>|. Assume 
current maximum modulus b Mm. For basis b(k,D at position p. if coirespondine residual 
r(k4.p) satisfies jlr(k.I,p)||||b(k.l)||<=Mm. then v s. 

I<r(k,l.p).b(k.l)>| <=||r(k,l,p)|| ||b(k.l)I| <=Mm 
Therefore, it is unnecessary to calculate inner product <r(k,l,p).b(k,l)> In this case, and 
the region r(k.l,p) is moved to next position. The nonn of basis |jb(k.I)|| can be calculated 
a priori (actually most of basis are normalized, namely ||bOc.l)||=l), the only oveiiiead for 
tfiis test to is to calculate en«gy of r(k.l,p). Fortunately, there is a fast algorithm to Kct 
||r(k.l.p)||. as described below. ^ 

Assunw thoe are n differmt sizes of basis heights {vi, va v„}. and m different sizes 

of basis widths {hi. hz. .... h„}, that are increasingly ordered. TTie search rectangle 
dimension is hs*ws. and the left-top point of the search rectangle is p(x,y). The 
hs*ws*n*m energy values can be calculated through die following four steps: 

Step 1: Calculate the energy for die s=hm+k columns (PIG. 11 shows an example of the 
columns). These columns are c«itered at (x-h,„/2+i.y), i«80.1.....s-l. Thdr height is vi. 
Their energy is represented as C,fl(0),Ci.,(0)....C,^(0), and calculated as 

Ci.i(0) =e(x-h™/2+l,y-vi/2) +...4e(x-ha/2+i.y)+...+ e(x4W2+i,y+vi/2) 
where eOtjr) r^resents the energy of pixels at position (xjr). 

The energies for the next s columns with same coordinates as above strips and length vz 
can be computed as 

CzM = Ci,i(0) + Extra (vz-vi) Pixels Energy, i=l A -.s 
G«ierally. we have: 

CiM = q.,^(0) + Extta <vi-vfi.|)) Pixels Energy, i=lA...s; j=l,2....n 

Step 2: Calculate energy of columns that are vertical shift of columns in Step 1. 

Cij(a)= Cj.i(a-1)- e(x-hm/2+i,y-vi/2+a-l) +e(x-h,„/2+i,y+vi/2i-a).a=l,...,hs 
where a represents the vertical shift number corresponding to y. 

Step 3: Calculate the energies of regions with height vj, (j=l,...,n) and width hi. hz, .... 
hm and center (x.y+a). (va=0,l hs) using 

%|(0,a)=:Cj.<i,ro*l)«(a)+...4<::jj„,tf2(a)+...4Cj.(hm*M 

Sj.2(0,a)= Sj.i(0.a)+ Extra (hz -h|) columns' energy 

Generally, 
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Sja(0.a)= Sj.i.i(0»a)+ Extra (hi - hoo)) columns* energy, 1=1,... ,m 

Step 4: Calculate the energies of the first set of regions with vertical base length vj, 0=1. 

n) and horizontal base length hi, (i=l,..,m) and center (x+b,y+a), <b=l,...,ws and 
a=:l hs) using 

Sj4(b,a)= Sj,i(b-l,a) - Cj.(hni.bi)/2+b-i(a)+Cj.aun+iu)/2+b(a) < 

The maximum modulus will be updated successively during the matching pursuit 
process; this will progressively confine the search space and hence, the name progressive 
elimination algorithm. Note that several bases can have the same sizes, ihus one energy 
calculation may avoid several inner product calculations. The performance of PEA is also 
related with how fast a good match (not necessarily the best match) is found. Because 
large regions always contain more energy, it is supposed to test bases of larger dimension 
first. 

If ||r(k,l,p)||>Mm, block 86 is executed to calculate the inner product (p) between rflc. I, p) 
and b(k,l). Block 87 compares die absolute value of p with current maximum modulus 
Mm. If |p|>Mm, the new Mm is set as |pi and the corresponding basts index and position 
are recorded. Regardless, we keep returning to block 84 until all search positions have 
been checked. Then blocks 83 through 88 are run repeatedly until all basis candidates 
have been tested. Finally, an atom is produced which includes three parameters: I. The 
index of basis in the dictionary that gives the best match; 2. The location of the best 
match in residual picture with (x, y) coordinates; and 3. The inner product (p) between 
basis and residual picture. FIG. 12 shows an example of an atom on a residual image. 

Returning to FIG. 4 again, the step after finding an atom is to record the atom parameters 
(block 63). Note in this stage, no quantization of the atom's modulus is p^ormed. 
Decision block 64 will decide when to begin atom quantization- Its operation depends on 
the rate control goal defined by the video compression system. If the compression ratio is 
fixed, block 64 will check if bits are still available for more atoms* Because no actual 
coding has been done yet. the used bits for coding the current atoms has to be estimated. 
Lei "Bip" represent the average bits for coding basis index and positions, "BmCi)'* 
represents the actual bits for the / 'th atom's modulus without quantization. Allocating one 
bit for the sign of inner product (p), then the used bits for n atoms are estimated as 

Used Bits = n*(Bip+l) +2(Bm(I) + Bm{2) + ...Bm(n)) 
where **Bfp'* is initialized according to experiential data for first residual fiame, and set as 
real value of last frame. Bm(i) can be known exactly for each modulus. An important fact 
is that the modulus will be quantized later and will result in fewer bits to be used than 
currently estimated. Hius in this stage we will get fewer atoms than what we can code. If 
the video system wants to achieve a certain quality, which is defined by the mean square 
error (MSE) of the coded residual as compared to the actual residual, block 64 will 
compare the current MSE achieved with the MSE objective. The MSE after introducing 
one atom is updated according to follow equation: 

MSE(n) = MSE(n.l) - p(n)*p(n) 
where MSE(n) represents the MSE after using n atoms and p(n) represents the inner 
product of nth atom, tailially the MSE, or MSE(0), is set to the energy of original 
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QS= 



lesidual picture. After quantization is performed, MSE(n) will likely increase, and 
therefore no longer achieve Ae MSB objective. In summary, if bits are available or the 
quality goal has not been achieved, the residual picture will be updated based on the 
current atom (block 65), followed by a search for another atom recommencing at block 
61. Otherwise if the bit or quality objective has been achieved; block 66 is executed for 
the quantization design. Residual updating, one step for the standard matching pursuit 
algorithm, can be described mathematically as 

r(k,l,p) = r(k,l,p) - p(n)*b(k,l) 
All regions not covered by current atom will be unchanged. 

The design of the quantizer (block 66) is based on minimum modulus (Minm) value 
found so far. The quantization step size (QS) is set to 

32 ifMinm>24; 
16 if2<Mumi<=24; 
8 if6<Minm<»12 
4 ifMinm<=6; 

All atoms found up to this point, will be quantized using the above QS in the simple mid- 
lead scalar quantization scheme. Next the residual picture is updated again accotding to 
the now quantized list of atom moduli 67. Assume that die atom coefficient before and 
after quantization are p(i), q(i) respectively (i«l,...,n). Assume that the corresponding 
bases are b(i), (i='l,...,n). The residual after n unquantized atoms is: 

E(n) = (Original Residual) - p(l)b(l) - p(2)b(2) p(n)b(n) 
Its energy ||E(n)|| is known also. There are two ways to calculate the residual eneigy after 
quantization. The first way is to simply calculate the residual after quantimtion as 

EQ(n) = (Original Residual) - q(l)b(l) - q(2)b(2) q(n)b(n) 
Another way is to update it recursively. Assume the quantization error for p(i) is ^p(i). 
Then the residual widi only p(n) being quantized is 

EQ(l) = E(n)-Ap(n)b(n)and||EQ(l)I| = ||E(n)|| + Ap(n)* Ap(n) - 2Ap(n)<E(n), b(n)> 
The residual with p(n),p(n-I) quantization becomes: 

EQ(2)«EQ(l).Ap(n-l)g(n.l) 
This relationship is true recursively and can be written as 

EQ (i) - EQ (i-1) - Ap(n-i+l)g(n-i+l).i-l,2,...n, EQ(0) - E(n) 
The conresponding energy is 

II EQ(i)I| HI EQ(i-l)l| + Ap(n4+1) Ap(n-i+l) - 2*Ap(n-i+l)< EQ(i-l),g(n-i+l)> 
Finally, we will get EQ(n) and ||EQ(n)||, which is the start point for further atom finding. 
An important tiling is that the list of atoms can be in any order for the recuisive update to 
occur - the update does not need to occur in the ord^ in which the atoms were found. 

Because the moduli of atoms have been quantized, more atoms will now be necessary to 
achieve the rate control or quality objective. Therefore, block 68 is executed to find 
additional atoms. The process is the same as block 61 through 63. However, the atom 
moduli will be quantized immediately in this stage. We now need to deal with atoms 
whose moduli is smaller than (QS - QS/4), without throwing them out by setting their 
quantization value to zero. The scheme used is given below: 



1 . If the atom modulus is larger than (QS - QS/4) then quantizer it using QS; 
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2. Otherwise^ if the atom modulus is laiger than (QS/2-QS/8} then it is quantized as 
value QS/2; 

3. Otherwise, if the atom modulus is larger than (QS/4-QS/16) then it is quantized as 
value QS/4; 

4. Otherwise, if the atom modulus is larger than (QS/8-QS/32) then it is quantized as 
value QS/8. 

Three levels down are good enough in practice alfliough more levels can, in theory be 
used. 

After block 68, a real rate control logic unit is executed (block 69). Because the atoms are 
quantized in-loop in this stage, the achieved quality or actual number of bits used can be 
estimated. When the compression goal is achieved, the system will go into the atom 
encoder 42. Otherwise, the residual picture wUl be updated based on the quantized atom 
modulus and the system will return to block 68 to find next atom. For color video, a 
residual picture contains seveial channels. i.e. Y, U and V channels. The atom 
decomposer 40 wiU be used for each channel independently. With this scheme, each 
channel can have its own bit budget or desired qualiQr goal. There are certain bit 
allocation methods, whidi can be used to allocate bit budgets for the different channels. 

All the atoms are passed to the atom encoder 42 for output in a compressed fonn. This 
invention considers the atom distribution for each channel as a bi*value map, as 
illustrated in Fig. 13. The black pixels represent atoms in then- corresponding position, 
while the white pixels represent a lack of atoms in that position. A quadtree-like 
technique is used to encode the positions containing atoms. The other parameteis of each 
atom will be encoded after the atoms position information, using variable length coding. 
The coding procedure for the atom parameter signal is illustrated in FIG. 14 and 
described in more detail below. 

The first step of atom encoding is to decompose the whole atom map (Fig. 13) into n*n 
blocks (Block 101). The value n may either be 16 (for the Y channel) or 8 (for the U and 
V channels). For each n*n block, if there are no atoms in the block, a zero-bit is output; 
otherwise, a one-bit is output, and die block is processed further to locate the atoms to the 
decoder. A quadtree decomposition procedure is used for this, and is summarized in the 
following four steps: 

Step 1. Initialize a list of atom blocks (LAB) with one element - the n*n block itself. 

Step 2. Pick one element e from LAB. If e's size is 1*1. output all atom parameters 
except for the position: namely the basis index, modulus and the sign of the inner 
product of e should be output, then proceed to step 4; otherwise, proceed to step 3. 

Step 3. Output the atom pattern bits of the four sub-blocks of c: 01820384, where ai 
(1=1,2,3,4) is one if there is an atom in die corresponding sub-block and zero 
otherwise. Put all sub-blocks f with ai value equal to 1 into the end of the LAB 
and return to step 2. 

Step 4. Check if the LAB is empty. If it is not empty, return to step 2; olherwise the 
encoding finishes for the one n*n block. 
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The basis index and atom modulus should be coded using a variable lengdi coder to 
conserve bits, since these signal parameters are not uniformly distributed The atom 
posinon information is encoded implicitly by recording the decomposition procedure 
with the 0/1 bit data. It is important to use a variable length coding method to encode the 
atom pattern bits of the four sub-blocks: ala2a3a4. There are 15 kinds of patterns for the 
atom pattern bits. ala2a3a4 (note 0000 is impossible). However, some patterns, such as 
1000, occur with much higher probabUity than other patterns. The probability of die 
different patterns can be estimated through experiments and used to create a variable 
length table design. Further, it should be noted that the probabUity distribution is variable 
for different channels and different atom densities. Therefore muWpie tables are used 
and the block's category information is encoded fiist so the decoder knows which table 
should be used (or decoding purposes. 

FIG. 15 illustrates the atom decoder 46. which performs operations that are the inveise of 
those performed by the atom encoda- 42. Kist, atom decodo- 46 receives one bit 
representing the status for the current n*n block. If die value is one. it is processed 
through the symmetric quadtree decomposition procedure. Initially, die n*n block is 
divided into four sub-blocks. The atom pattern bits for the four sub-blocks are decoded 
usmg the inverse variable length coding (VLC). Then all the sub-blocks with value 1 are 
put into list of atom blocks (LAB). The LAB is updated dynamically by decomposmg 
each element in the LAB recursively and getting its atom pattern bits. If an element from 
the LAB is a 1*1 block, die atom basis index and the modulus should be decoded using 
the inverse VLC tables: the bit representing the sign of the inner product should then be 
read in. The atom decode for one n*n blodc is finbhed if the LAB becomes empty. 

The decoded atom parameter signal is dien passed to the residual re-constructor 48, 
which forms the residual picture one channel by one channel using die method of 
classical matching pursuit Initially the residual image is set to be O's. Then each atom is 
added one by one using the foUowuig procedure. Let q(i) and b(i, k. I) represent the i'th 
atom coefficient and the corresponding 2D basis matrix respectively. If (x(i) y(i)) 
represents the location of the i'th atom, then the matrix q(i)*b(i, k, I) is added to the 
rraidual image constructed so far at position <x(i). y(i)) to get the new current residual. 
The process repeats until all atoms have been added for the channel. Once each channel 
has been decomposed, the process is finished and die residual image has been 
reconstructed. 



Those familiar with the previous matching pursuit based video coding art will recognize a 
number of advantages associated with the disclosed techniques. Hie atom decomposition 
process based on an over-complete basis space has been speeded up significanUy through 
a more accurate energy region esUmation procedure and through the progressive 
candidate elimination algorithm. The atom modulus quantizer design is seamlessly 
chosen by the atom decomposition scheme, while the previous art specified the quantizer 
before the transfonnation began. Finally, die atom encoding process is more efTicient 
because spatial relatioaships between the atoms are exploited by the invented quadtree- 
based decomposition scheme. The previous art collects all atoms into a ID list, makine it 
hard to effici«itly code them. 
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